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SUMMARY 


A  two-process  model  of  pattern  discrimination  was  developed 
to  describe  how  tonal  sequences  are  processed,  stored,  and 
discriminated  by  the  human  auditory  system.  In  this  model,  the 
comparison  of  auditory  stimuli  involves  processsing  in  two 
separate  concurrent  modes,  with  the  mode  contributing  the  least 
variance  having  the  dominant  effect  on  performance,  in  the 
'sensory  trace'  mode,  the  subject  compares  a  trace  of  the 
auditory  sensation  to  the  corresponding  trace  of  a  subsequent 
stimulus.  The  trace  is  assumed  to  decay  with  time,  but  be 
independent  of  the  number  and  range  of  possible  stimuli,  in  the 
'context  coding'  mode,  stimuli  are  discriminated  by  comparing 
encoded  representations,  each  representation  consisting  of  a 
comparison  between  the  given  stimulus  and  the  overall  context  of 
other  stimuli  heard.  The  quality  of  the  encoded  representation  is 
assumed  to  depend  on  the  size  of  the  auditory  context. 

This  model  was  applied  to  tasks  in  which  a  subject  compared 
two  sequences  of  tones  and  judged  whether  the  frequency 
patterns  were  the  same  or  different.  Several  aspects  of  the 
sequences  were  varied:  (a)  the  temporal  variability  of  the 
sequences,  (b)  the  correlaton  between  the  temporal  envelopes  of 
each  pair  of  sequences  presented  on  a  trial,  and  (3)  the  time 
interval  between  the  two  sequences.  In  the  'correlated' 
condition,  the  two  sequences  on  a  trial  had  identical  temporal 
patterns,  while  in  the  'uncorrelated'  condition,  all  sequence 
temporal  patterns  were  generated  independently.  In  order  to 
study  the  properties  of  the  trace  mechanism,  various  temporal  and 
spectral  transformations  were  made  to  the  second  sequence  of  each 
pair.  In  addition,  maskers  consisting  of  short  tonal  sequences 
were  Interposed  between  the  first  and  second  sequence  on  each 
trial. 

Performance  in  the  correlated  conditions  was  independent  of 
temporal  variability,  but  decreased  with  increases  in  the  time 
Interval  between  the  sequences.  Performance  in  the  uncorrelated 
condition  decreased  with  increased  temporal  variability,  and  was 
independent  of  the  length  of  the  intersequence  interval.  These 
results  support  the  assumptions  of  the  trace/context  theory;  good 
fits  were  obtained  between  data  from  human  observers  and  the 
predictions  of  the  two-process  model.  Performance  in  the 
correlated  condition  was  Insensitive  to  temporal  transformations 
made  to  the  second  sequence,  suggesting  either  (a)  that  traces 
can  be  temporally  scaled  for  comparison  with  other  stimuli,  or 
(b)  that  sensory  traces  can  be  generated  for  separate  aspects  of 
a  stimulus.  Performance  under  different  frequency 
transformations  produced  a  pattern  of  results  consistent  with 
(b) .  The  effects  of  interfering  events  occurring  during  the  time 
interval  between  the  sequences  were  not  consistent  with 
conventional  assumptions  about  interruption  or  masking  of  the 
trace  mechanism. 

A  series  of  experiments  on  temporal  pattern  discrimination 
was  begun  in  which  the  observer  had  to  discriminate  between  two 
tonal  sequences  having  the  same  or  different  temporal  patterns. 

ii 


These  tasks  were  evaluated  under  different  levels  of  pattern 
difference  and  average  duration,  and  under  different  amounts  of 
temporal  compression  and  expansion.  Performance  was  found  to  be 
very  sensitive  to  the  temporal  transformation  Imposed  on  the 
stimuli.  Models  of  temporal  pattern  discrimination  are  currently 
being  evaluated  in  a  revised  task  paradigm. 

Some  general  assumptions  of  the  signal  detection  model  were 
evaluated  in  experiments  on  the  detection  and  recognition  of 
multiple  element  auditory  and  visual  displays.  A  major  Interest 
in  these  experiments  was  to  det^cribe  how  the  observer  aggregates 
information  over  the  elements  of  the  sequentially  or 
simultaneously  presented  elements  of  the  display.  The  results 
were  consistent  with  expectations:  (a)  in  sequential  auditory 
displays,  first  and  last  arriving  elements  are  emphasized,  (b) 
with  visual  displays,  elements  in  the  fixated  region  are 
emphasized  (the  effective  size  of  the  region  depends  on  the 
element  coding  and  format) ,  (c)  with  both  auditory  and  visual 
displays,  high  information-carrying  elements  are  emphasized,  and 
(d)  with  the  visual  displays  thus  far  evaluated,  the  basic 
relationships  predicted  by  the  recognition-detection  model  are 
supported . 
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1.  INTRODUCTION  AND  PURPOSE 

The  goal  of  this  research  is  to  understand  how  human 
listeners  process  and  store  brief  sequences  of  sounds. 

Specifying  how  humans  encode  and  remember  such  stimulus  patterns 
is  an  important  goal  for  a  theory  of  auditory  information 
processing,  and  is  a  necessary  requirement  for  understanding 
complex  perceptual  processes  such  as  speech.  We  have  developed  a 
general  paradigm  for  studying  the  memory  processes  involved  in 
the  perception  of  auditory  patterns,  using  tasks  in  which  the 
temporal  and  spectral  properties  of  tonal  sequences  are 
systematically  manipulated. 

1.1  Basic  Experimental  Paradigm 

When  a  subject  must  judge  the  similarity  of  the  frequency 
patterns  defined  by  two  sequences  of  tones,  variation  in  the 
timing  of  the  tones  can  have  a  deleterious  effect  on  performance. 
This  effect  is  most  evident  when  the  tonal  sequences  have  unique 
temporal  patterns.  Under  such  conditions,  it  may  be  difficult 
for  the  subject  to  tell  whether  or  not  the  frequency  patterns  are 
the  same  or  different.  Our  experiments  have  demonstrated  that 
variability  in  the  temporal  aspects  of  a  tonal  sequence  has  a 
large  effect  on  a  subject's  ability  to  make  discriminations  based 
on  the  frequency  pattern  of  the  sequence.  We  have  employed  this 
sequence  discrimination  task  to  evaluate  a  model  of  human  pattern 
discrimination,  and  have  studied  some  properties  of  the  memory 
process  and  candidate  mechanisms  of  the  sequence  comparison 
process . 

Many  Investigators  have  noted  that  varying  the  parameters  of. 
a  tonal  sequence  can  have  a  large  effect  on  performance  in  a 
discrimination  or  detection  task.  For  example,  Watson  and  his 
colleagues  have  reported  a  niuober  of  studies  in  which  a  subject 
must  detect  small  differences  between  two  tonal  sequences  whose 
pareuneters  vary  across  the  sequence  of  experimental  trials.  This 
type  of  trial>by-trlal  variation  can  have  an  effect  on  the 
detectability  of  a  small  change  in  the  frequency,  duration,  or 
intensity  of  one  component  of  the  tonal  sequence  (Watson,  Kelly 
&  Wroton,  1976;  Watson  &  Kelly,  1981;  Spiegel,  &  Watson,  1981; 
and  Leek  &  Watson,  1984) .  Watson  (1987)  has  pointed  out  that 
these  effects  can  be  much  larger  than  those  obtained  from  single 
tone  (2IFC)  discrimination  tasks  in  which  the  tone  parameters 
vary  over  trials. 

We  reported  similar  effects  in  an  experiment  testing  the 
discrimination  of  temporal  jitter  in  tonal  sequences  (Sorkin, 
Boggs  &  Brady,  1982) ,  and  in  more  recent  frequency  pattern 
discrimination  experiments  in  which  the  temporal  pattern  was 
varied  (Sorkin,  1987b) .  The  subject's  task  in  the  recent 
experiments  was  a  Same/Different  task:  subjects  were  presented 
with  a  pair  of  tonal  sequences  and  had  to  determine  if  the  two 
sequences  had  the  same  or  different  frequency  patterns. 

Svibjects  were  to  ignore  variations  in  the  temporal  structure  of 
the  sequences,  such  as  jitter  in  the  tone  onsets,  durations,  or 
inter-tone  gaps . 
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Figure  1  Illustrates  some  of  the  tonal  sequences  possible 
on  trials  in  our  sequence  experiments.  In  condition,  C,  the 
correlated  temporal  condition,  the  pattern  of  tone  onsets, 
durations,  and  inter-tone  gaps  was  identical  in  the  two  sequences 
of  each  trial.  For  this  particular  pair  of  sequences  the 
frequency  pattern  is  different  in  the  two  sequences,  so  that  the 
correct  response  is  "different"  on  this  trial.  The  second  pair 
of  sequences  Illustrates  condition  U,  the  uncorrelated  condition. 
Here  the  tone  onsets,  durations,  and  gaps  were  uncorrelated 
between  the  two  sequences.  The  frequency  pattern  is  identical  on 
the  trial  shown,  hence  the  correct  response  is  "same".  The 
temporal  variability  of  the  sequences  was  the  major  experimental 
variable  in  this  paradigm:  in  the  correlated  condition,  the 
temporal  variation  was  only  present  across  trials;  in  the 
uncorrelated  condition,  the  variation  was  present  both  within  and 
across  trials. 
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Figure  1.  Examples  of  pairs  of  sequences  present  on  trials 
of  the  experimental  task. 
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There  are  other  ways  that  the  temporal  properties  of  the 
sequences  could  be  varied.  For  example,  the  last  pair  of 
sequences  Illustrates  the  synchronized  (S)  condition.  In  this 
condition  the  tone  onsets  were  identical  in  the  two  sequences, 
but  the  durations  and  gaps  varied.  In  this  particular  pair  of 
sequences  the  frequency  pattern  is  identical,  so  the  correct 
response  is  "same".  In  another  condition  (figure  5)  all 
durations  and  gaps  in  the  first  sequence  of  each  pair  were  scaled 
upward  by  a  constant  factor,  so  that  the  new  sequence  was  an 
expanded  version  of  the  first.  In  all  tasks,  the  siibjects  were 
told  to  base  their  response  only  on  the  tonal  pattern  of  each 
sequence . 

2.  TWO-PROCESS  PATTERN  DISCRIMINATION  THEORY 

In  a  standard  type  of  signal  detection  task,  detectability, 
d',  is  proportional  to  signal  power,  I,  and  to  the  square  root  of 
the  signal  duration,  t.  It  is  Inversely  proportional  to  the 
square  root  of  the  sum  of  two  variances:  The  external  noise 
power  density,  and  any  internal  noise  or  sensory  variance 

Equation  1  illustrates  this  relationship. 

d'  -  kl  t/[^n2+  (1) 

The  detectability  is  also  a  function  of  various  memory  parameters 
that  are  specific  to  the  particular  task.  For  example,  a  single¬ 
interval  yes-no  detection  task  clearly  has  different  memory 
requirements  than  does  a  two-interval  intensity  discrimination 
task. 


Tanner  (1961)  proposed  a  general  framework  for  categorizing  . 
the  memory  requirements  of  a  variety  of  single  and  two-interval 
psychophysical  t2isks;  this  scheme  has  been  extended  and 
quantified  by  a  number  of  workers  including  Sorkin  (1962),  and 
Macmillan,  Kaplan,  and  Creelman  (1977)  in  a  study  of  categorical 
perception.  Tanner's  model  included  a  short-term  decaying  memory 
for  the  acoustic  input  to  the  system  plus  various  interference 
factors  and  long-term  memory  factors.  The  trace  decay  component 
of  the  Tanner  model  was  extended  in  a  paper  by  Kinchla  &  Smyzer 
(1967)  and  later  Incorporated  as  the  trace  component  of  the 
trace-context  model  of  Durlach  and  Braida  (1969)  and  their 
colleagues  (Berliner  and  Durlach,  1973).  In  Sorkin  (1984),  we 
proposed  that  the  Durlach  and  Braida  dual  mode  model  could  be 
extended  to  the  sequence  discrimination  task. 

According  to  the  Durlach-Braida  model,  a  subject  can  employ 
two  different  processing  modes  in  a  discrimination  task:  a  trace 
mode  and  a  context  mode.  In  the  trace  mode,  the  comparison 
operation  is  performed  on  internal  memory  traces  of  the  input 
signals.  The  trace  is  a  direct  representation  of  the  acoustic 
input,  e.g.  a  precategorlcal  replica,  and  it  deteriorates  or 
picks  up  noise  over  time.  The  longer  the  time  interval  between 
two  Inputs  to  be  discriminated,  the  more  use  of  the  trace  mode 
will  lead  to  degraded  performance.  In  the  context  mode,  each 
input  is  first  categorized  into  one  of  a  defined  set  of  codes. 

The  comparison  operation  is  then  performed  on  these  encoded 
representations.  Once  encoded,  the  Internal  data  are  not  subject 
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to  degradation  over  time.  However,  there  is  a  context  variance 
present  which  is  a  function  of  the  difficulty  of  the  encoding 
process.  Task  variables  such  the  stimulus  range  are  assumed  to 
affect  this  context  noise.  Performance  in  a  given  task  will 
usually  Involve  trade-offs  in  the  internal  noise  associated  with 
each  mode  of  operation. 

These  considerations  are  incorporated  into  the  Durlach  and 
Braida  models  in  the  following  manner: 

2  -2  -2  -1  1/2 

d'  -  c  /  [  ^  +  (  ^  +  f  )  ]  (2) 

Is  c  t 

where  c^  is  a  constant  in  our  tasks  incorporating  factors  such  as 
the  detectability  and  discriminability  of  the  elementary  tones 
(in  a  minimum  uncertainty  task) ,  and, 

2  2  2 

[f  ,  (t  t  and  (T  are  variance  components  added  to  the 
sc  t 

discrimination  process  as  a  consequence  of  the  sensory,  context, 
and  trace  aspects  of  the  task,  respectively.  It  can  be  seen  from 
an  examination  of  equation  2  that  the  trace  and  context  variances 
will  operate  together  in  determining  which  process,  or  whether 
either  process,  will  have  a  large  effect  on  perfomance.  For 
example,  if  the  variance  associated  with  one  process  is  much 
smaller  than  the  variance  associated  with  the  other  process,  the 
smaller  process  will  have  the  dominant  effect. 

The  sensory  variance  refers  to  the  noise  from  internal  and 
external  sources  other  than  those  associated  with  the  encoding 
and  storing  of  the  sequence  information.  In  the  sequence 
discrimination  paradigm  the  sensory  variance  is  assumed  to  be 
small  relative  to  the  context  and  trace  variances.  The  context 
variance  is  assumed  to  be  a  function  of  the  number  of  different 
sequence  patterns  that  may  be  present  on  a  trial.  The  trace 
variance  is  assumed  to  be  a  function  of  the  time  period  over 
which  the  information  from  the  first  sequence,  e.g.  the  trace 
datum,  must  be  held  in  order  to  make  a  comparison  with  the 
information  from  the  second  sequence. 

Consider  the  discrimination  model  applied  to  the  conditions 
in  the  sequence  comparison  tasks  of  the  present  study.  Suppose 
that  the  two  sequences  of  a  trial  have  identical  temporal 
envelopes,  as  in  the  correlated  condition  of  figure  1.  A  direct 
comparison  of  the  traces  of  the  two  sequences  will  be  a  good 
strategy  for  discriminating  any  difference  in  the  frequency 
patterns,  provided  that  the  time  period  between  the  sequences  is 
not  excessive.  In  the  uncorrelated  condition,  when  the  two  input 
sequences  do  not  share  identical  temporal  patterns,  the  use  of 
the  trace  mode  may  not  be  effective.  This  is  because  there  are 
now  differences  between  the  traces  of  the  two  inputs  that  are  not 
relevant  to  the  frequency  discrimination  task.  These  irrelevant 
differences  are  associated  with  the  decorrelation  of  the  temporal 
envelopes  of  the  two  sequences.  We  would  expect  these 
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differences  to  result  in  added  variance  in  the  trace  comparison 
process.  This  reasoning  led  us  to  define  the  trace  variance  as 
follows: 

2  2 
^  -  c  [ISI  +  nd  +  (n-l)g]  +  c  f  (  ,  g,  . . . )  (3) 

t  2  3  g 

where  C2  and  C3  are  constants,  ISI  is  the  inter-sequence- 
interval,  n  is  the  number  of  tones  in  each  sequence,  d  is  the 
average  duration  of  each  tone,  g  is  the  average  duration  of  the 
gaps  between  the  tones, is  the  gap  standard  deviation,  and 
f(  )  is  a  function  describing  the  added  variance  due  to  the 
decorrelation  of  the  temporal  envelopes  of  the  sequences  to  be 
discriminated.  This  function  incorporates  the  effect  of 
irrelevant  (i.e.  not  frequency-contour)  differences  between  the 
sequences  on  a  trial.  The  effects  of  such  differences  will  be 
minimal  when  context  mode  processing  is  dominant. 

2.1  Empirical  Tests  of  the  Two-process  Model 

Predictions  of  specific  versions  of  the  model  were  evaluated 
in  a  series  of  sequence  comparison  experiments  reported  in  Sorkin 
(1987b) .  The  results  of  these  experiments  are  summarized  in 
figures  2  through  4.  Figure  2  shows  the  average  performance  of 
three  subjects  in  the  conditions  Illustrated  in  figure  1.  The 
Independent  variable  is  the  standard  deviation  of  the  time  gap 
between  tones.  The  smooth  curves  are  model  fits  to  the  data  of 
the  uncorrelated  and  correlated  condition.  Performance  in  the 
correlated  condition  in  these  experiments  is  generally 
independent  of  gap  variability;  performance  in  the  uncorrelated  . 
conditions  drops  to  an  asymtotic  value.  These  results  have  been 
obtained  in  a  number  of  different  experiments  using  different 
rules  to  generate  the  different  frequency  patterns  over  trials 
and  different  frequency-difference  manipulations.  Performance  in 
the  synchronized  condition  increases  slightly  as  a  function  of 
gap  variability. 

Figure  3  shows  performance  as  a  function  of  the  average  gap 
duration,  for  a  fixed  level  of  gap  variability.  Performance  in 
the  uncorrelated  condition  increases  as  a  function  of  duration, 
while  performance  in  the  correlated  condition  drops  to  the  same 
value.  The  smooth  curves  shown  in  the  figure  are  model 
predictions  based  on  parameter  fits  to  data  in  a  separate  set  of 
conditions  in  which  the  mean  gap  duration  was  fixed  and  the  gap 
variability  was  varied.  Two  points  about  figure  3  should  be 
noted:  First,  the  effect  of  the  relative  gap  variability, 
is  mainly  on  performance  in  the  uncorrelated  condition  (from 
figure  2  we  know  that  gap  variability  does  not  affect  performance 
in  the  correlated  condition) .  Second,  performance  in  the 
correlated  condition  drops  at  high  gap  durations,  because  of  the 
effect  of  the  total  sequence  duration  on  the  trace  decay  time. 
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Figure  2.  Average  performance  (plotted  points)  as  a  function 
of  the  gap  standard  deviation  and  condition  (C,S,U) 
(8  tones,  mean  gap  100  ms) .  Solid  lines  are 
theoretical  functions. 


Mean  Gop  Duration  (ms) 


Figure  3.  Average  performance  (plotted  points)  as  a  function 
of  the  mean  gap  duration  and  condition  (12  tones, 
gap  standard  deviation*40  ms,  ISI^O.5  s) .  Solid 
lines  are  theoretical  functions  based  on  parameter 
fits  to  data  obtained  with  8  tone  sequences  at  a 
mean  gap  of  100  ms  and  a  range  of  gap  variabilities. 
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Figure  4.  Average  performance  (plotted  points)  as  a  function 
of  Intersequence  interval  (ISI) ;  same  subjects  and 
conditions  as  figure  2  (8  tones,  mean  gap^SOms,  gap 
standard  devlatlon’-30  ms) .  Solid  lines  are 
theoretical  functions  based  on  parameter  fits  to 
the  data  of  figure  2. 
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The  difference  between  performance  in  the  correlated  and 
uncorrelated  conditions  is  also  apparent  when  examined  as  a 
function  of  the  intersequence  interval  shown  in  figure  4; 
performance  in  the  correlated  condition  drops  over  intersequence 
intervals  of  from  500  to  3000  ms.  Performance  in  the 
uncorrelated  condition  is  essentially  independent  of  ISI  (or 
drops  much  more  slowly) .  It  is  evident  that  the  two-component 
discrimination  model  expressed  in  equations  2  and  3,  captures  the 
essential  aspects  of  the  results  obtained  to  date.  The  effects 
of  gap  duration  and  variability,  intersequence  interval,  and 
correlation,  on  sequence  discrimination  performance  are  all 
consistent  with  the  model. 

The  analysis  of  sequence  discrimination  behavior  into  two 
particular  areas:  the  sensory  trace  process,  and  the  sequence 
comparison  process.  In  addition,  we  have  broadened  the  paradigm 
to  include  the  discrimination  of  temporal  jitter  as  a  function  of 
the  spectral  aspects  of  the  patterns.  The  objective  was  to 
better  describe  the  trace  process  and  to  assess  its  sensitivity 
to  certain  types  of  transformations  (such  as  linear  time  scaling) 
and  to  potentially  Interfering  inputs  (such  as  irrelevant  sounds 
occurring  during  the  inter-sequence- interval)  . 

2.2  Experiments  on  the  Trace  Mechanism 

An  important  question  in  sequence  comparison  is  the 
sensitivity  of  the  trace  and  context  mechanisms  to  particular 
types  of  transformations  in  the  sequences  to  be  compared.  The 
context  mechanism  is  assumed  to  be  insensitive  to  the  length  of 
the  ISI  and  to  the  effects  of  certain  events  occurring  during  the 
ISI  period.  The  trace  mechanism,  on  the  other  hand,  is  assumed 
to  be  highly  sensitive  to  the  duration  of  the  ISI  and  to  certain 
other  potentially  interfering  events,  such  as  the  occurrence  of 
similar  stimuli  during  the  ISI  period.  The  specific  effects  of 
other  types  of  transformations  of  the  sequences  are  of 
considerable  theoretical  Interest.  Linearly  scaling  the  duration 
of  the  Initial  or  final  sequence,  for  example,  should  have  a 
different  effect  on  performance  while  in  the  two  modes.  Such  a 
scaling  may  or  may  not  Impose  additional  processing  demands  on 
either  mode. 

2.2.1  Duration  Scaling 

One  interesting  question  concerns  whether  duration 
compression  or  expansion  will  produce  decrements  in  the  trace 
mode  which  are  a  function  of  the  ISI.  That  is,  will  duration 
scaling  interact  with  the  increase  in  trace  variance  over  time? 
The  answer  depends  on  one's  concept  of  the  trace  process.  If  the 
trace  is  a  true  raw  representaion  of  the  input,  then  compressing 
or  expanding  the  sequence  should  degrade  performance  in  a  fashion 
similar  to  the  addition  of  temporal  jitter  to  the  sequence 
Intervals,  and  an  interaction  will  result.  If,  however,  the 
trace  is  (effectively)  only  a  partially  encoded  representation  of 
the  tone  bursts  and  the  temporal  (rhythmic,  etc.)  relationships 
among  the  tone  bursts,  then  such  transformations  may  have  a  quite 
different  effect.  In  that  case,  the  effect  of  such  a 
transformation  would  not  produce  interactions  with  ISI,  but 
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rather  would  resemble  the  effects  of  adding  temporal  jitter  in 
the  context  mode:  e.g.  produce  a  fixed  decrement,  depending  on 
the  magnitude  of  the  transformation. 

We  performed  some  experiments  testing  these  predictions 
(Sorkin  and  Snow,  1987) .  In  one  experiment,  we  modified  the 
manner  In  which  the  durations  of  the  tones  and  gaps  of  the  second 
sequence  In  a  trial  were  generated.  In  the  'normal'  duration 
transformation  condition,  the  second  sequence  in  a  trial  was 
generated  without  modification.  Trials  in  this  condition  were  of 
the  same  type  as  those  shown  earlier.  In  the  'expanded'  duration 
transformation  condition,  the  durations  of  all  tones  and  gaps  in 
the  second  sequence  of  the  trial  were  multiplied  by  a  factor  of 
1.4,  producing  a  sequence  40%  longer  than  it  would  have  been 
without  the  transformation. 

Duration  transformation  was  manipulated  factorially  with 
temporal  correlation,  producing  the  'correlated-expanded' 
condition,  in  which  the  two  sequences  in  a  trial  had  the  same 
temporal  pattern  prior  to  expansion  of  the  second  sequence,  and 
the  'uncorrelated-expanded'  condition,  in  which  the  temporal 
patterns  of  the  two  sequences  in  a  trial  were  different  prior  to 
expansion  of  the  second  sequence.  Typical  trials  from  each  of 
these  conditions  are  shown  in  figure  5. 

We  were  Interested  in  whether  operation  in  the  trace 
mode  requires  that  the  two  sequences  have  correlated  temporal 
envelopes.  Another  way  to  put  the  question  is  to  ask  whether 
subjects  would  treat  expansion  of  the  second  sequence  of  a  trial 
as  a  difference  in  the  sequence  temporal  patterns-  similar  to  that 
found  in  the  uncorrelated  condition-  or  if,  instead,  they  would 
be  able  to  scale  the  duration  of  the  sensory-trace  of  one 
sequence  to  match  that  of  the  other.  If  such  a  scaling  process 
were  possible,  then  one  would  expect  to  see  the  same  general 
pattern  of  results  in  the  correlated-expanded  condition  as  in  the 
correlated-normal  condition.  If  such  a  scaling  process  were  not 
possible,  then  the  pattern  of  performance  in  the  correlated- 
expanded  condition  should  closely  resemble  that  found  in  the 
uncorrela  ;ed-normal  condition. 

The  results  can  be  seen  in  figure  6  and  7.  Figure  6  shows 
perfoinnance  averaged  across  subjects  for  the  correlated 
conditions.  Performance  in  both  the  normal  and  expanded  cases 
was  Independent  of  gap  standard  deviation,  but  decreased  with 
increasing  ISI.  Figure  7  shows  performance  in  the  uncorrelated 
conditions.  In  this  condition,  performance  was  independent  of 
ISI  in  the  conditions  involving  large  differences  in  sequence 
temporal  patterns  within  a  trial  (i.e.  those  in  which  the  gap 
standard  deviation  was  40) ,  but  decreased  with  ISI  in  those 
conditions  which  did  not  Involve  large  temporal  pattern 
dlffemces  (i.e.  those  in  which  the  gap  standard  deviation  was 
10) .  Note  that  performance  in  both  the  uncorrelated-normal  and 
uncorrelated-expanded  conditions  decreased  with  increasing  gap 
standard  deviation.  The  results  from  the  'normal'  duration 
transformation  conditions  replicate  the  results  of  our 
earlier  experiments  while  the  results  from  the  'expanded' 
conditions  support  the  proposed  duration-scaling  hypothesis. 
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Although  performance  in  the  expanded  conditions  is  poorer  than 
that  in  the  normal  conditions,  perhaps  due  to  trace-decay  during 
duration  scaling,  the  general  pattern  of  results  is  the  same, 
leading  one  to  believe  that  S's  were  able  to  compensate  for 
sequence  expansion  in  these  conditions,  perhaps  by  "scaling"  the 
sensory  traces. 

2.2.2  Frequency  Transformation 

A  second  question  related  to  trace  transformations  concerns 
the  degree  to  which  the  trace  mechanism  is  sensitive  to  frequency 
transformations.  Because  of  the  octave  generalization 
phenomenon,  we  would  expect  that  the  trace  would  be  insensitive 
to  octave  frequency  transformations,  but  not  to  other 
transformations.  An  experiment  was  run  in  which  the  observers 
had  to  make  a  same/different  judgment  on  the  frequency  contour  of 
the  sequence.  In  this  task  the  definition  of  a  correct  response 
is  very  sensitive  to  the  observer's  understanding  of  the 
instructions;  therefore,  one  should  be  cautious  in  interpretating 
the  results.  Four  intersequence  interval  conditions  were  rvm 
(500,  1000,  2000,  and  4000  ms.)  with  four  frequency 
transformations  (0%,  10%,  50%,  and  100%) .  Performance  dropped 
from  a  d'  of  about  2.5  with  0%  transformation  to  a  d'  of 
approximately  1.0  with  a  100%  transformation;  performance  dropped 
about  0.8  d'  units  between  the  500  and  4000  ms.  intersequence 
Intervals.  There  was  no  significant  interaction  between 
frequency  transformation  and  intersequence  interval,  indicating 
that  observers  can  not  scale  frequency  in  the  same  way  as 
duration. 

2.2.3  Masking 

Another  experimental  manipulation  studied  in  the  context  of 
trace  vs.  context  mode  processing,  was  the  effect  of  interfering 
sound  Inputs  which  occur  during  the  intersequence  interval. 

Other  investigators,  such  as  Masarro  and  his  colleagues  (Kallman 
and  Massaro,  1983)  have  argued  for  the  occurrence  of  an 
interruption  in  forming  and  consolidating  information  from  the 
pre-contextual  trace.  The  correlated-uncorrelated  jitter 
paradigm  allows  evaluation  of  the  differential  effects  of  such 
potentially  Interfering  signals.  That  is,  the  interfering  signal 
should  not  only  produce  effects  on  performance  dependent  on  the 
time  interval  between  the  offset  of  the  sequence  and  the  onset  of 
the  signal,  but  should  also  produce  effects  which  demonstrate  a 
shift  from  trace-mode  like  processing  to  context-mode  like 
processing.  These  effects  should  be  observable  as  a  differential 
dependence  of  performance  on  the  gap  '^variability  and  the  ISI. 

An  experiment  to  evaluate  this  hypothesis  was  performed 
using  8-tone  tonal  sequences  similar  to  those  described 
previously  (at  frequencies  of  500,  909,  1667,  and  2857  Hz.) 
equated  for  perceived  loudness  (at  SPLs  of  70,  70,  68,  and  64 
dBA,  respectively).  Tone  bursts  were  50  ms.  long  with  linear 
onset  and  offset  envelopes  of  2  as.  The  gap  durations  were  drawn 
from  a  normal  distribution  (approximate)  with  a  mean  of  50  as  and 
a  standard  deviation  of  either  10  as  or  40  as.  The  Intersequence 
intervals  were  1000,  2000,  and  4000  as.  The  frequency  pattern  of 
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the  first  sequence  contained  two  of  each  of  the  four  possible 
frequencies  in  a  random  order;  if  the  trial  was  a  'different' 
trial,  the  frequencies  of  the  second  or  third,  fourth  or  fifth, 
and  sixth  or  seventh  tones  in  the  second  sequence  (selected 
randomly)  were  changed  to  one  of  the  three  remaining  frequencies. 
The  correlated  and  uncorrelated  manipulations  were  the  same  as 
described  in  the  previous  experiments. 

In  the  masked  conditions,  a  three-tone  sequence  was  played 
during  the  ISI,  starting  350  ms  after  the  end  of  the  last  tone  in 
the  first  sequence.  The  duration  of  each  tone  in  the  mask  was  50 
ms.  Mask  gap  durations  were  drawn  from  an  approximation  to  a 
normal  distribution  with  a  mean  of  50  ms.  In  the  'fixed'  mask 
condition,  the  standard  deviation  of  this  distribution  was  0  ms. 
In  the  'random'  mask  condition,  the  standard  deviation  of  this 
distribution  was  25  ms.  Tones  in  the  'fixed'  mask  were  1231  Hz, 
at  a  level  of  70  dBA.  Tones  in  the  'random'  mask  were  selected 
at  random  from  the  set  of  four  which  comprised  the  eight-tone 
comparison  sequences,  subject  to  the  constraint  that  no  frequency 
could  occur  more  than  once  in  the  same  mask.  Trials  in  the 
normal  phase  consisted  of  a  500  ms  visual  ready  signal,  a  500  ms 
wait,  the  first  sequence,  the  variable  ISI,  the  second  sequence, 
and  observer  response  followed  by  trial  feedback.  Trials  in  the 
mask  phase  were  the  same  except  for  the  occurrence  of  the  mask 
during  the  ISI  (the  total  ISI  was  the  same) . 

Observers  had  extensive  experience  in  the  non-mask 
conditions,  from  having  participated  in  previous  experiments  on 
sequence  discrimination.  In  the  mask  phase  of  the  experiment, 
temporal  correlation  and  mask  type  were  held  constant  during  each 
sessions  (6  blocks  of  100  trials),  while  gap  standard  deviation 
and  ISI  were  manipulated  factorlally  across  blocks  within  each 
test  session.  The  order  of  ISI  and  gap  deviation  was  random. 
There  were  8  testing  sessions,  each  observer  participating  in  two 
blocks  from  each  of  the  24  experimental  conditions  produced  by 
the  combination  of  levels  of  temporal  correlation,  mask  type,  gap 
deviation,  and  ISI. 

As  in  our  previous  experiments,  only  performance  in  the 
uncorrelated  conditions  decreased  with  increasing  gap  standard 
deviation.  Performance  In  the  random  mask  condition  was  lower  in 
both  the  correlated  and  uncorrelated  conditions  than  performance 
in  the  normal  or  fixed-mask  conditions.  However,  performance  in 
the  masked-correlated  conditions  was  superior  to  that  in  any  of 
the  uncorrelated  conditions.  Figure  8  shows  the  averaged 
results  as  a  fxmctlon  of  ISI  for  the  (Ts  40  ms  condition.  (The 
pattern  of  performance  obtained  in  the  correlated  and 
uncorrelated  conditions  for  (f  >  10  ms  was  essentially  the  same 
as  that  found  for  tr  •  40  ms.)  The  standard  error  of  each  point 
was  estimated  by  taking  the  square  root  of  the  mean  of  the 
variances  (Gourevitch  and  Galanter,  1967)  of  the  individual 
subject  d's.  The  effects  of  the  masker  appear  to  be  Independent 
of  the  correlated/uncorrelated  manipulation.  One  possible 
conclusion  is  that  the  'trace'  process  is  not  more  susceptible  to 
the  effects  of  an  intervening  masker  than  is  the  context  process. 


Figure  8.  Average  performance  as  a  function  of  intersequence 
Interval  with  and  without  intervening  masker  (gap 
standard  devlation«*40  ms) . 


2 . 3  Temporal  Pattern  Discrimination 

In  these  experiments,  the  observer  was  asked  to  disciminate 
between  two  tonal  sequences  which  had  the  same  or  different 
temporal  or  rhythmic .  rather  than  frequency  patterns.  The 
Initial  objectives  were  to  model  the  discrimination  process  using 
the  types  of  sequence  comparison  algorithms  developed  for 
frequency  pattern  discrimination  (e.g.  Sorkin,  1987a) .  In 
addition,  we  were  Interested  in  testing  for  the  trace/context 
distinctions  obsezn^ed  in  the  frequency  discrimination  task  and, 
in  addition,  in  evaluating  the  effects  of  temporal  transformation 
or  scaling  on  rhythmic  discrimination.  The  expectation  was  that 
discrimination  performance  would  be  highly  resistant  to  the 
temporal  scaling  manipulation,  over  a  range  of  time 
transformations . 

The  stimuli  and  conditions  were  similar  to  those  previously 
described.  Tone  sequences  consisted  of  8,  1000  Hz  tone  bursts  of 
50  ms  duration  at  75  dBA.  The  intersequence  interval  was  1000  or 
3000  ms.  The  gaps  were  approximately  Normal  with  a  mean  of  50  ms 
and  a  minimum  gap  of  5  ms.  On  'same'  trials,  the  durations  of 
the  gaps  in  the  second  sequence  were  Identical  to  those  of  the 
first.  On  'different'  trials,  jitter  was  added  to  the  duration 
of  each  gap  in  the  first  sequence  to  obtain  the  durations  of  gaps 
in  the  second  sequence.  The  amount  of  jitter  was  determined  by  a 
uniform  distribution  with  a  mean  of  0  ms  and  a  variable  standard 
deviation,  ^  diff*  During  the  first  phase  of  the  experiment,  a 
psychometric  function  of  ^iff  was  obtained,  shown  in  figure  9. 
The  value  of  ^dlff  vas  varied  from  ll  to  40  ms  (varied  across 
blocks  of  trials  and  sessions) .  Performance  increased  with  ^dlff 
in  the  expected  fashion. 

ISI  and  temporal  transformation  were  varied  during  the  next 
phase  of  the  experiment;  temporal  scaling  was  accomplished  by 
multiplying  each  tone  and  gap  duration  in  the  second  sequence  by 
a  duration  transformation  parameter  following  normal  sequence 
generation.  The  jitter  was  added  to  sequences  in  'different' 
trials  prior  to  the  duration  transformation.  There  were  seven 
duration  transformation  conditions  (varied  across  blocks)  ranging 
from  0.4  (maximum  compression)  to  1.6  (maximum  expansion). 
Observers  were  Informed  prior  to  each  session,  which 
transformation  condition  they  were  running,  ^dlff  held 
constant  at  30  ms;  the  order  of  presentation  of  the  duration 
transformation  condition  was  random  (but  with  no  repeats) .  An 
additional  set  of  conditions  were  run  in  which  the  first  sequence 
in  each  trial  were  also  duration-transformed;  in  other  words, 
additional  control  conditions  were  run  in  which  the  average  onset 
to  onset  Intervals  were  held  constant  within  a  trial  and  block. 
These  effectively  non-trans formed  conditions  enable  us  to  factor 
out  the  effect  of  duration  on  the  duration  transformation  task. 

The  results  of  the  temporal  transformation  manipulation  are 
shown  in  figures  10  and  11.  The  top  curve  of  figure  10  Indicates 
how  temporal  discrimination  Improves  with  net  total  average 
duration  of  the  sequence,  for  a  fixed  ratio  of  ^iff  to  tone+gap 
duration.  The  lower  curve  shows  the  effect  of  transformation  in 
the  condition  when  the  second  sequence  is  transformed  relative  to 
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the  first.  Figure  11  shows  the  effect  of  transforming  the  second 
sequence  when  the  effect  of  duration  is  factored  out.  It  can  be 
seen  that  the  effect  of  temporal  transformation  is  relatively 
symmetric  about  zero  and  that  the  resulting  function  is  highly 
peaked.  Little  effect  of  ISI  can  be  observed  on  performance. 

This  sensitivity  to  expansion  or  contraction  was  unexpected  and 
is  not  consistent  with  our  current  models  of  the  discrimination 
process  (especially  in  light  of  the  lack  of  an  effect  of  ISI) . 
Because  of  the  presence  of  across-trial  variation  in  the  gaps, 
and  because  of  the  manner  in  which  the  difference  jitter  was 
added,  this  discrimination  task  is  relatively  complicated  to 
define  from  an  ideal  observer's  point  of  view.  Accordingly,  we 
have  modified  the  task  somewhat  in  our  current  experiments. 

3.  DETECTION  AND  RECOGNITION  OF  MULTIPLE  ELEMENT  SIGNALS 

In  addition  to  these  experiments  on  sequential  auditory 
Inputs,  we  have  continued  to  gather  data  on  detection  with  multi¬ 
element  auditory  and  visual  signals.  These  experiments  are 
designed  to  evaluate  how  information  relevant  to  a  detection 
decision  is  aggregated  over  multiple  display  elements  (either  in 
a  sequence  of  display  elements,  or  in  a  simultaneous  presentation 
of  display  elements) .  We  reported  some  of  these  results  in 
Sorkin  et  al.  (1987)  in  Sorkln,  Mabry,  and  Weldon  (1988),  and  in 
MaUary  (1987) .  Experiments  have  continued  on  two  questions:  (a) 
the  effects  of  varying  the  statistics  of  the  displayed  elements 
over  the  spatial  position  of  the  display,  and  (b)  whether  the 
recognition-detection  model  (Green  and  Blrdsall,  1978;  Green, 
Weber,  and  Duncan,  1977)  is  valid  for  the  multiple-element 
situation.  The  data  acquisition  phase  of  this  project  was 
completed;  initial  analysis  Indicates  support  for  the  extension 
of  the  recognition-detection  model. 
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Figure  11.  Average  performance  in  the  temporal  discrimination 
task  as  a  function  of  the  transformation 
factor.  The  data  in  the  lover  curve  of  figure  10 
have  been  corrected  for  the  effect  of  duration. 
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